fix(tls12): reject oversized AEAD plaintext - #22

What this PR changes

Closes #17.

This PR makes the TLS 1.2 AEAD decrypters reject records whose decrypted plaintext is larger than the TLS record-size limit.

The new check is applied in both TLS 1.2 AEAD decrypt paths:

• AES-GCM;
• ChaCha20-Poly1305.

If decryption succeeds but the plaintext length is greater than 16 KiB, the provider now returns Error::PeerSentOversizedRecord.

What is the TLS record-size limit?

TLS does not send an entire connection as one giant encrypted blob. It splits traffic into records. In TLS 1.2, the plaintext carried by one record is limited to 2^14 bytes, which is 16 KiB.

RFC 5246, the TLS 1.2 specification, says the record layer carries data in chunks of 2^14 bytes or less and that the TLSPlaintext length must not exceed 2^14: https://datatracker.ietf.org/doc/html/rfc5246#section-6.2.1

So after a TLS 1.2 record is decrypted, the provider should not hand rustls a plaintext record larger than that limit.

Why this matters

This crate plugs into rustls as a crypto provider. The provider is responsible for decrypting TLS records and returning plaintext messages back to rustls.

The built-in rustls providers enforce this limit in their TLS 1.2 AEAD decrypters. For comparison:

• rustls ring TLS 1.2 provider source: https://docs.rs/rustls/latest/src/rustls/crypto/ring/tls12.rs.html
• rustls aws-lc-rs TLS 1.2 provider source: https://doc.servo.org/src/rustls/crypto/aws_lc_rs/tls12.rs.html
• rustls fragment limit constant is 16 KiB / 16384 bytes: https://rustdocs.bsx.fi/src/rustls/msgs/fragmenter.rs.html

Before this PR, rustls-cng-crypto decrypted the record and returned the plaintext without this provider-side length check. That could make this provider accept an authenticated TLS 1.2 record that rustls' built-in providers would reject.

Why this fix is the right thing to do

The fix follows the same pattern as rustls' built-in TLS 1.2 providers:

1. authenticate and decrypt the record;
2. compute the plaintext length;
3. if the plaintext length is greater than 16 KiB, return Error::PeerSentOversizedRecord;
4. otherwise continue returning the plaintext.

The error variant matters. PeerSentOversizedRecord tells rustls this was a peer protocol violation, not a local CNG failure and not an authentication-tag failure.

How to read this if you are not a crypto expert

This PR is about enforcing a size rule, not changing encryption itself.

Imagine TLS records as sealed envelopes. AEAD decryption verifies the seal and opens the envelope. This PR adds a check after opening: "Is the paper inside larger than TLS allows for one envelope?" If yes, rustls rejects the record even though the cryptographic tag was valid.

That matches the TLS 1.2 spec and rustls' own providers.

Validation

This PR adds a regression test that:

1. encrypts a TLS 1.2 plaintext of MAX_FRAGMENT_LEN + 1 bytes;
2. decrypts it through the provider;
3. verifies that decryption returns Error::PeerSentOversizedRecord.

The test covers both AES-GCM and ChaCha20-Poly1305 TLS 1.2 AEAD paths.

A follow-up commit fixed a test-module import issue caught by CI. The intended behavior is unchanged.